Learn to cycle: Time-consistent feature discovery for action recognition

Generalizing over temporal variations is a prerequisite for effective action recognition in videos. Despite significant advances deep neural networks, it remains challenge to focus on short-term discriminative motions relation the overall performance of an action. We address this by allowing some flexibility discovering relevant spatio-temporal features. introduce Squeeze and Recursion Temporal Gates (SRTG), approach that favors inputs with similar activations potential variations. implement idea novel CNN block uses LSTM encapsulate feature dynamics, conjunction gate responsible evaluating consistency discovered dynamics modeled show consistent improvement when using SRTG blocks, only minimal increase number GFLOPs. On Kinetics-700, we perform par current state-of-the-art models, outperform these HACS, Moments Time, UCF-101 HMDB-51.1